At present, the face recognition methods based on deep learning have the problems of large model parameter size and slow feature extraction speed, and the existing face datasets have the problem of single pose, which cannot achieve good recognition effect in the actual face recognition task. Aiming at this problem, a multi-pose face dataset was established, and a lightweight multi-pose face recognition method was proposed. Firstly, the MTCNN (Multi-Task cascaded Convolutional Neural Network) algorithm was used by the method for face detection, and the high-level features included in the last network of MTCNN were used for face tracking. Then, the face pose was judged according to the positions of the detected face key points, the current face features were extracted by the neural network with ArcFace as loss function, and the current face features were compared with the face features of the corresponding pose in the face database to obtain the face recognition result. The experimental results show that the accuracy of the proposed method is 96.25% on the multi-pose face dataset, which is 2.67% higher than that on the face dataset with single pose. It shows that the proposed multi-pose face recognition method can effectively improve the recognition accuracy.